Papers with inner mechanisms
Unfolding the External Behavior and Inner Affective State of Teammates through Ensemble Learning: Experimental Evidence from a Dyadic Team Corpus (L18-1)
Copied to clipboard
Aggeliki Vlachostergiou, Mark Dennison, Catherine Neubauer, Stefan Scherer, Peter Khooshabeh, Andre Harrison
| Challenge: | a study aimed to understand the relationship between the external behavior and inner affective state of two team members during a demanding operational task. |
| Approach: | They assessed the external behavior and inner affective state of two team members during a demanding operational task. |
| Outcome: | The study compared verbal responses of two team members to their external and internal cues . the results suggest that an association exists between turn taking and inner affective state . |
Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing defense methods focus on detecting harmful prompts or reducing the likelihood of harmful responses. |
| Approach: | They propose a layer-specific editing method to align LLMs to harmful prompts by supervised fine-tuning and reinforcement learning. |
| Outcome: | The proposed method improves the performance of large language models against jailbreak attacks while maintaining performance on benign prompts. |
Trigger-Argument based Explanation for Event Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing works on ED use words or phrases to explain models’ inner mechanisms, but for ED, the event structure is more enlightening clues to explain model behaviors. |
| Approach: | They propose a Trigger-Argument based Explanation method which can utilize event structure knowledge to uncover a faithful interpretation for existing ED models at neuron level. |
| Outcome: | The proposed method can reveal the process by which the model predicts on the large-scale MAVEN and the widely-used ACE 2005 datasets. |
Inserting Information Bottlenecks for Attribution in Transformers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained transformers are a popular approach for understanding features important for prediction. |
| Approach: | They apply information bottlenecks to analyze attribution of features for prediction on a black-box model. |
| Outcome: | The proposed method outperforms two competing methods in degradation tests on four datasets. |
Diagnosing Vision-and-Language Navigation: What Really Matters (2022.naacl-main)
Copied to clipboard
Wanrong Zhu, Yuankai Qi, Pradyumna Narayana, Kazoo Sone, Sugato Basu, Xin Wang, Qi Wu, Miguel Eckstein, William Yang Wang
| Challenge: | Existing models claim to be able to align object tokens with specific visual targets, but there are non-negligible gaps between the two. |
| Approach: | They conduct diagnostic experiments to examine how the agents perceive multimodal input by ablation diagnostics input data. |
| Outcome: | The results show that indoor and outdoor navigation agents refer to object and direction tokens when making decisions. |
What Part of the Neural Network Does This? Understanding LSTMs by Measuring and Dissecting Neurons (D19-1)
Copied to clipboard
| Challenge: | Biological neural systems consist of a huge number of neurons, and can react to the environment in complicated ways. |
| Approach: | They propose a metric to quantify the sensitivity of neurons to each label and conduct experiments to prove it. |
| Outcome: | The proposed metric is based on a set of experiments that show that dropping an arbitrary neuron significantly degrades the accuracy of the model. |
Evaluating and Enhancing the Robustness of Code Pre-trained Models through Structure-Aware Adversarial Samples Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained code models have made significant strides in the field of neural code intelligence, but they are susceptible to adversarial attacks that subtly modify the input sequence and can impair generalization. |
| Approach: | They propose a set of novel robustness evaluation methods based on the intrinsic structure of the code to explore the impact of imperceptible perturbation. |
| Outcome: | The proposed methods have demonstrated their effectiveness across a wide range of models and tasks, and are able to predict the performance of perturbed models. |
Explicit Bayesian Inference to Uncover the Latent Themes of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive generative capabilities, yet their inner mechanisms remain largely opaque. |
| Approach: | They propose a variational autoencoder-based neural topic model to interpret LLMs generation process through an explicit Bayesian framework by inferring latent topic variables via variational inference. |
| Outcome: | The proposed model outperforms state-of-the-art topic models on intrinsic measures of coherence and diversity on multiple datasets and shows significant gains on classification and summarization tasks. |